1. Field of the Invention
The present invention relates to the field of computer systems used to index and retrieve digitized images, digitized sounds, and related data, including social expression card designs, multimedia presentations, and the like.
2. Statement of the Problem
Computer systems that allow users to view images stored in digital form have been used in various computer applications for many years. Recently, interactive computer-based systems have been developed that allow consumers and others who are not computer literate to view images, including images of various products with associated text, and hear sounds such as speech and music, as part of a purchase process, educational process, or other activity. Often, retrieval of entities to be displayed by such systems requires the user to select descriptive text, symbols, or other items displayed in a series of hierarchical menus or scrolling lists. After one or a series of menu items have been selected, the corresponding images and related information are retrieved and displayed for the user.
Several computer-based systems that allow images to be retrieved by using a menu system and displayed have been invented in the past, including the following:
______________________________________ Inventor U.S. Pat. No. Issue Date ______________________________________ Kita et al. 5,172,245 Dec. 15, 1992 Cannon 5,056,029 Oct. 8, 1991 Buckley et al. 5,036,472 July 30, 1991 ______________________________________ "CreataCard" system was introduced by Custom Expressions, Inc., Glendale, California, in May 1990.
U.S. Pat. No. 5,056,029 was previously issued to one of the present inventors. This patent discloses a system for manufacturing and vending social expression cards that queries the customer to input a series of special occasion parameters to select and display a group of cards from a database of available card designs.
The "CreataCard" system was introduced by Custom Expressions, Inc., in May 1990. This system is an on-site unit for drawing and vending personalized social expression cards that uses a menu to allow customers to display a series of card images on a touch-sensitive display screen. After choosing the desired card, the customer enters information to personalize the card, such as names, dates, ages, messages, etc.
Kita et al. disclose an electronic information retrieval apparatus for image information that uses primary and secondary menus.
Buckley et al. disclose a computer-controlled machine for vending printed products, including personalized greeting cards and the like. The user selects the type of product desired from a displayed menu. The unit provides audio and video presentations of the products and options available to the customer. Base products such as preprinted forms are stored within the unit for transfer by a robotic arm to a printer for customization.
There are also a number of commercially available software programs that allow personal computer systems to print greeting cards. "Card Shop" by Artsci, Inc., allows Macintosh computer users to select from a menu of approximately fifty greeting cards covering a variety of occasions.
Social expression cards include greeting cards, invitations, announcements, and the like and are referred to herein as "greeting cards" or "social expression" cards. Current greeting card systems usually require the user to first select a menu item representing a general card category, such as "Birthday" or "Sympathy," from a displayed menu of general categories. Selection is typically accomplished by using a touch screen or other input device. After selecting a general category (e.g., "Birthday"), the user is prompted to select one subcategory from each of several additional sequentially displayed menus to further define the particular type of image the user is interested in viewing. The menu displayed after each selection is determined by the menu item previously selected and is typically made up of items that can be considered a logical subcategory of the menu item previously selected. For example, logical menu items that might be presented in a greeting card design retrieval system after selection of "Birthday" would be "Friend" and "Relative." If the menu item "Relative" is selected, then a menu with the items "Father," "Mother," "Brother," and "Sister" would logically be presented. After the last menu item is selected (e.g., "Brother"), one or more greeting card images, having characteristics defined by the menu items previously selected (i.e., "Birthday", "Relative" and "Brother") are displayed for examination by the consumer.
Image retrieval systems, such as current greeting card systems, typically have a large number of images available for users to view. Each image has a number of characteristics including, for example, the character and content of graphic elements of the image and the character and content of text elements displayed with the image. The use of a hierarchical menu to limit displayed images to those of particular interest to a user is a more efficient means to locate such images than traditional means, such as, in the case of greeting cards, racks in retail outlets that display thousands of cards and typically group cards in broad categories by publisher.
One significant problem in using a menu in the manner described is that a predetermined hierarchical menu structure necessarily means that the menu items available for selection are limited in number and that the sequence in which menu items can be chosen is fixed. Since a user may have a very personal way of conceptualizing the important characteristics of a desired greeting card design, the menu items presented may include characteristics that are not important to the user, the menu items presented may not include characteristics that are important to the user, and the menu item words or symbols used to represent specific characteristics may not be the same as those which the user has in mind to describe the same characteristics.
These problems can be overcome in part by increasing both the number of items available on each menu and the number of menus presented before entities, such as greeting card designs, are displayed or presented. However, beyond a certain point, both the time required to traverse the menus presented and the difficulty a user may have in choosing between menu items perceived as unimportant or irrelevant defeat the purpose of using menus. The purpose is to quickly present a user with entities, such as greeting card designs, whose characteristics match those desired by the user.
The most common alternative method used to search image databases is accomplished by attaching lists of words, phrases, characters, or symbols that describe or represent image characteristics ("keywords") to each image file and then performing searches by entering one or more keywords to retrieve those images that have matching keywords.
A computer-based system that allows images to be retrieved by using keywords is discussed in U.S. Pat. No. 5,107,343 of Kawai, issued on Apr. 21, 1992. Kawai discloses an information searching system for image data in which a character or symbol is used as a keyword for a still or moving picture in order to display a still or moving picture associated with a piece of music.
There are a number of commercially available software programs that allow computer systems to retrieve various types of graphic data by using keywords, as discussed in the article entitled "Managing Your Media," MacUser (September 1993, pages 190-206). Users of these programs must first enter descriptive keywords that are attached to each graphic item before the item data can be retrieved by using the keywords. Subsets of one or more graphic data items can then be retrieved by entering one or more of the keywords previously associated with the data items. For example, the program "ImagePals" (U-Lead Systems, Inc., 970 West 190th Street, Suite 520, Torrance, Calif. 90502) allows miniature representations of images ("thumbnails") to be retrieved and displayed by entering graphic data attributes in the form of a text description attached earlier to each graphic data item.
Searching for graphic data items by using more than one attribute or keyword typically requires the use of the logical or Boolean operators AND, OR, and NOT. The program "NeoView" (Neotech Systems, Inc., 103 West 61st Street, Westmont, Ill. 60559) allows various types of images, including document images and photographs, to be retrieved in several ways including free-form entry of previously indexed keywords or phrases input by the user and associated with images.
A major problem with this method of using keywords to retrieve entities is that a user must use the same keywords to describe an entity as were used in the initial indexing process. While a user might guess some keywords that should be used, there is no way for a user to be confident that the keywords he or she might use to describe the characteristics of an entity will in fact retrieve all the available entities that have the characteristics the user is attempting to describe.
One method of solving this problem is to restrict the keywords that can be used to retrieve entities to a specific set of keywords ("controlled keyword vocabulary"). For example, U.S. Pat. No. 5,167,011 of Priest, issued on Nov. 24, 1992, discloses a method to coordinate the storage and retrieval of information by using controlled vocabulary terms to link functions. Controlled keyword vocabularies are widely used in information retrieval systems used to retrieve text documents. Various methods to improve the effectiveness of such systems have been evaluated including the use of some type of thesaurus so the terminology of an indexer can be reconciled with that of a searcher. The ideal thesaurus for use in such a system is considered by some to be one containing any term applied by any user to an object or entity where each of the terms is used as a referent to that object or entity. It has been demonstrated experimentally that while any one individual can normally think of only a few synonymous terms to describe an entity, the total number of different terms used by a group of persons to describe the same entity will be quite large. Another problem occurs when a word has more than one meaning ("polysemy") and therefore is used to refer to a number of entities that are not alike; for example, the word "stamp" could refer to either a postage stamp or a rubber stamp.
A number of computer-based systems for storing and retrieving related linguistic expressions (including synonyms) from a large database have been invented in the past, including the following:
______________________________________ Inventor U.S. Pat. No. Issue Date ______________________________________ Squillante et al. 5,007,019 Apr. 9, 1991 Kaji et al. 4,916,614 Apr. 10, 1990 Rosenbaum et al. 4,384,329 May 17,1983 ______________________________________
Squillante et al. disclose an electronic thesaurus that enables a user to track through a series of related words to obtain an extensive set of related meanings.
Kaji et al. disclose a system for performing automatic language translation comprising a first memory to store a source language analysis dictionary, a second memory to store a target language generation dictionary, a third memory to represent co-occurrence relation, a thesaurus memory to represent implication among concepts, a source language input device, a target language output device, and a processor to translate a source language sentence into a target language sentence by utilizing the first, second, third, and thesaurus memories.
Rosenbaum et al. disclose a storage method and control system for storing and interactively accessing a large database of related linguistic expressions such as synonyms and antonyms.
3. Solution to the Problem
None of the prior art describes a method and apparatus that allows graphic, sound, and related data, such as greeting card designs, to be rapidly and efficiently indexed and retrieved by using freely chosen keywords where the keywords as well as equivalent words, such as synonyms, metonyms, misspellings, and foreign language equivalents of the keywords, are automatically linked to the images.